Applied Issues in Treatment Outcome Assessment

It is an exciting time to conduct alcoholism treat­ ment outcome evaluation. Advancements in statis­ tical software for personal computers, for example, have dramatically increased the type and complexity of techniques available to the evalua­ tor. Although some concern has been raised about how the democratization of the tools of evaluation may precipitate their inappropriate use (e.g., Pedhazur 1982), indirect evidence suggests that increased accessibility has had an overall positive effect in the field. Miller et al. (1995a) found, for example, that the methodological quality of research outcome studies has improved signifi­ cantly in the past 20 years, much of this due to selection of assessment instruments with known psychometric properties and the appropriate use of multivariate techniques. Software advances for personal computers have also spawned an audiencefriendly revolution in how findings are presented, with time-to-event outcomes, hierarchical linear modeling findings, and structural equation model­ ing findings now presented in an understandable and graphic format. It is also a critical time for doing rigorous outcome evaluation. In many States evaluation is now legislatively mandated, with future program appropriations tied to demonstration of treatment effectiveness. Programs and jobs can hinge on how well an evaluation report communicates find­ ings to audiences unfamiliar with research methodology and the multifaceted nature of alcohol treatment outcome(s). Under these condi­ tions the evaluator has a clear responsibility to select assessment tools with demonstrated reliabil­ ity and validity that are also sensitive to, and theo­ retically consistent with, treatment program objectives. The purpose of this chapter is to familiarize the reader with a variety of fundamental issues that arise in the conduct of outcome evaluation in alcoholism treatment. The relative merits of specific measures of alcohol consumption (see the chapter by Sobell and Sobell) and biological markers (see the chapter by Allen et al.) are reviewed elsewhere in this Guide and will not be reiterated. This chapter begins with a general discussion of the importance of using assessments with strong psychometric properties. Reliability theory is described from an applied perspective, with examples provided using assessment tools reviewed in this Guide. The next section briefly addresses the goals of summative and formative alcohol-related outcome evaluation, highlighting the differences between individual and groupbased evaluation. This is followed by a section that reviews alternative perspectives of alco­ holism, with attention directed to how these defin­ itions of alcoholism suggest relevant measures of change; a section that discusses the measurement of behavior change across time, noting how commonly observed patterns of behavioral change differ across particular domains of functioning;